Coding image data

ABSTRACT

A system encodes video frames in accordance with the MPEG standard, in real-time. Motion vectors are calculated during the forward prediction of B frames and P frames using a telescoping procedure. Similarly, a telescoping procedure is used for calculating motion vectors for B frames during backwards predictive coding. The use of telescoping for the backward predictive coding results in B frames being coded in the wrong order. The correct order for B frames is re-established by writing the frames to a buffer in the order in which they are generated and then reading the frame data from the buffer in the order in which they are required for transmission or recording purposes. The use of the telescoping procedure reduces significantly the amounts of computation required for calculating motion vectors during predictive coding for frames which are displaced by more than one frame period. The use of a buffer to reorganize the frames does not add significantly to the overall hardware requirement because such a buffer would often be used for smoothing out irregular data rates.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation-in-part of our commonly assignedapplication Ser. No. 08/165,693 filed Dec. 13, 1993 now abandoned.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method of and apparatus forpredictive coding for video frames.

2. Related Art

Predictive coding is a known technique for compressing video data. Avideo frame to be coded is compared with a reference frame and, ratherthan generating data indicative of the frame in isolation, dataindicative of the differences between the reference frame and the frameto be coded are generated.

Comparing individual pixels between a reference frame and a frame to becoded becomes significantly less efficient if movement occurs within theimage. However, given that movement often results in a re-arrangement ofelements within an image, rather than a complete re-generation of animage, there is still a degree of inherent redundancy and advantage maybe taken of this in order to facilitate further compression.

Movement within a video sequence often consists of an element beingmoved between different portions of an image. Given that video framesare presented at 25 or 30 per second, the degree of movement, for mostobservable objects, is relatively modest from one frame to another.Thus, as previously stated, movement often consists of there-arrangement of pixel values and advantage may be taken of thisinherent redundancy in order to facilitate further compression.

In theory, it would be possible to consider each individual pixel and,for each individual pixel, calculate a motion vector which, for apresent frame under consideration, identifies the position of a pixel ina reference frame having a similar value. However, it will beappreciated that the generation of motion vectors for each individualpixel would provide little compression compared to transmitting theindividual pixel values themselves, therefore, in order to achieve adegree of compression, it is necessary to calculate motion vectors forpixel regions and to assume that the whole region moves in a similarway. It is therefore known to divide video images into blocks of pixelsconsisting of, for example, squares made up of 16×16 pixels. Thus,before difference data is transmitted for each pixel within a blockunder consideration, a search is performed around a similar region in areference frame to identify a block of pixels in the reference framewhich, when compared with the block to be transmitted, results in, onaverage, the lowest absolute difference terms. Thus, once determined,these difference terms are transmitted, along with a motion vectoridentify the position of the identified block within the referenceframe, relative to the position of the block under consideration.

Predictive coding can be used to generate difference values for theframe (n+1) which immediately follows the reference frame (n). Inaddition, the method may also be used to predict subsequent frames, suchas frame n+2, n+3 etc. Under these circumstances the reference framesearch area, for which movement has to be considered, increases for eachincremental frame displacement. Furthermore, it should be appreciatedthat such an increase occurs in two dimensions.

In order to facilitate the calculation of motion vectors, it is possibleto perform a telescoping technique, in which a motion vector isidentified for the first frame displacement, between frame n and framen+1 whereafter a motion vector is calculated for frame n+2 withreference to frame n using the motion vector calculated for frame n+1 todefine a shifted detection range.

When coded frames are being transmitted in real time, it is desirable totransmit coded data as soon as possible. Thus, in the example givenabove, it is necessary to code frame n+1 in order that motion vectorsmay be calculated for frame n+2. Consequently, it is desirable, as soonas frame n+1 has been coded, to transmit this frame and thereafter toeffect processing for the coding of frame n+2 so that this frame may betransmitted.

In some situations, particularly when transmitting data in accordancewith an accepted standard, the desired order of transmission may differfrom that which naturally results from the telescoping process. Aproblem therefore results in that it is difficult to implement atelescoping process if the resulting coded frames are generated in thewrong order.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide an improved methodand apparatus for predictive coding for video frames.

It is a further object of the present invention to provide an improvedmethod and apparatus for predictive coding by telescoping, wherein theresulting coded frames are generated in an order which differs from thatrequired by a particular transmission standard.

According to a first aspect of the present invention, there is provideda method of the aforesaid type, characterised by writing frame n-1 to abuffer before writing frame n-2 to said buffer; and reading frame n-2from said buffer prior to reading frame n-1 from said buffer.

In a preferred embodiment, a frame n is predicted from an earlier framen-m by forward predictive coding.

According to a second aspect of the present invention, there is providedapparatus of the aforesaid type, characterised by buffering means; meansfor writing coded frame n-1 to said buffering means prior to writingcoded frame n-2 to said buffering means; and means for reading codedframe n-2 from said buffering means prior to reading coded frame n-1from said buffering means.

In a preferred embodiment, the apparatus codes frames in accordance withthe MPEG recommendation.

In a further preferred embodiment, the apparatus is used in videoconferencing facilities of video telephones.

BRIEF DESCRIPTION OF EXEMPLARY EMBODIMENTS

The invention will now be described by way of example only, withreference to the accompanying figures, in which:

FIG. 1 shows a system for compressing and decompressing video signals,for transmission or storage, including a circuit for effecting temporalcompression;

FIG. 2A illustrates a block of pixels of a frame to be coded and a blockof pixels in a reference frame, for the purpose of calculating a motionvector;

FIG. 2B illustrates an algorithm for calculating a motion vector;

FIG. 3 details a known circuit for effecting temporal compression, ofthe type shown in FIG. 1;

FIG. 4A illustrates types of frames considered in accordance with theMPEG recommendation, prior to coding;

FIG. 4B illustrates a coded stream of MPEG frames;

FIG. 5 details a circuit for decoding video frames compressed inaccordance with the MPEG recommendation;

FIG. 6 illustrates areas or a reference frame compared with a frame tobe coded, for calculating motion vectors, when frames are separated bymore than one frame spacing;

FIG. 7 illustrates a telescopic search for calculating motion vectors;

FIG. 8 shows a circuit for motion vector calculation with telescoping,including a circuit for calculating motion vectors; and

FIG. 9 details the motion vector calculation circuit, identified in FIG.8.

A system for compressing video signals, transmitting of storing saidsignals and decompressing said transmitted or stored video signals, isshown in FIG. 1. Full bandwidth video signals are generated by a videosignal generator 14, which may take the form of a video camera or videoreproducing means, such as a magnetic tape or an optical disc playeretc. The signal generator 14 generates a sequence of original frames 15which are supplied to a temporal compression circuit 16. The temporalcompression circuit takes advantage of similarities between temporallyspaced frames and generates coded signals representing the differencesbetween said frames. The output from said temporal compression circuit16 is supplied to a spatial compression circuit 17, arranged to takeadvantage of further redundancies within specific image frames. Thus,the spatial compression circuit 17 may make use of discrete cosinetransform coding, such that the amount of data generated for each frameis dependent upon the actual amount of information contained within theframe.

Thus, the combination of temporal compression and spatial compression isknown to achieve a significant degree of video data compression,allowing a compressed video signal of this type to be transmitted overlimited bandwidth channels, such as those used in video telephony or,alternatively, stored on standard computer-based storage media, such assingle-platen magnetic discs or audio-based compact optical discs(CD-ROMS).

The output signal derived from the spatial compression circuit 17 isamplified by amplifier 18 for transmission or storage, as identified byreference 19.

At a receiver, or within replaying apparatus, a coded signal receivedfrom a transmission medium or read from a storage device, is supplied toan amplifier 20, whereafter the procedures performed to achievecompression are effectively reversed, so as to decompress the picture.Thus, circuit 21 performs spatial decompression, whereafter a circuit 22performs temporal decompression, thereby supplying a full bandwidthvideo picture to a picture generating device 23, such as a videomonitor, which in turn displays a sequence of decompressed video frames24.

the present invention is particularly concerned with aspects of temporalcoding and decoding, although it should be appreciated that, in manyenvironments, spatial compression techniques will be included inaddition to temporal compression techniques. A compression techniquewhich does take advantage of both of these types of compression is thatproposed by the Moving Picture Experts Group of the InternationalStandards Organisation (ISO), commonly referred to as MPEG.

Temporal compression is effected by taking a first frame as a referenceframe and comparing this reference frame with a frame which is to becoded. Coding is achieved by identifying differences between the frameto be coded and the reference frame and coding a representation of thesedifferences, so that the original frame may be reconstructed at thereceiver or upon playback.

A first step would consist of comparing each pixel of the frame to betransmitted with the corresponding pixel of the reference frame.Generally, pixels are made up of several colour components, such as RGBor, more commonly in transmission systems, luminance plus two colourdifference signals. Thus, for each of these components, the value of theframe to be transmitted may be compared with the reference frame and,rather than transmitting the full value for the pixel to be transmitted,the difference between the two pixel values may be transmitted. Thus, insome situations, the difference between the two values on aframe-by-frame basis may be very small, particularly if the originatingimage is stationary. However, as movement occurs, the differences willstart to get larger and with substantial amounts of movement, thesavings may become minimal. Furthermore, given an environment havinglimited bandwidth for transmission or storage, movement may result inpicture break-up.

Although differential coding techniques will eventually break down ifsignificant movement occurs, it is apparent that, in the majority ofvideo sequences, movement consists of actual objects within an imageframe moving over a predetermined period of time. Thus, given that asignificant number of frames will elapse in just one second of the videosequence, it is likely that a moving object will not result insignificant changes to the overall frame information content, althoughsignificant changes will be occurring to individual pixels. Thus, itwould be possible to reduce the amount of information transmitted ifinformation identifying a motion vector could be transmitted, along withdifference signals taken, not from the original positions of thereference frame, but from positions shifted from the original inaccordance with the motion vector.

A system for compressing video data in this way, that is to say, by thecalculation of motion vectors in addition to differential coding, isdisclosed in U.S. Pat. No. 5,083,202, assigned to the present Assignee.In accordance with this disclosure, a video frame is divided into aplurality of blocks, each consisting of 16 lines with 16 pixels on eachline, that is, a 16×16 block. The division of frames into blocks is alsorequired for spatial compression, using the discrete transform methodbut, usually, this involves a division into blocks of 8×8 pixels. Whenusing both techniques, each 16×16 pixel region is assembled from four8×8 blocks and, to distinguish between the two, the larger block isreferred to as a macro-block.

A macro-block of pixels of this type is shown in FIG. 2A. Motion vectorsare not derived from, and are not necessarily related to, actualmovements of objects shown in the image frame. As used herein, a motionvector refers to a particular 16×16 macro-block and it represents amovement, in the positive or negative X and Y directions, by an integernumber of pixel spacings, of a block in the reference image which, whencompared with the block to be transmitted, produces the lowestdifference values. Thus, it is possible that a vector can be identifiedwhich satisfies this criterion without actually being related to anactual motion of an object in the image frame. However, in practice, itis likely that a vector will be identified which satisfies thiscriterion due to actual motions which have occurred substantially in thedirection of the motion vector.

The resolution of a motion vector is substantially an arbitrary choice,based upon the degree of compression required and the processingfacility available. In this example, the motion vectors are measured interms of whole pixel spacings, however, fractional pixel spacings couldbe used and new pixel values, at sub-pixel positions, could becalculated by interpolation. Similarly, the size of the search area hasbeen chosen so as to accommodate most speeds of movement, while at thesame time placing realistic constraints on the processing facilityrequired in order to calculate the motion vectors. This is particularlyimportant when the motion vectors, on the transmission side, are beingcalculated in real time.

The identification of the preferred motion vector for a particular blockis a highly computationally demanding procedure. However, it should beappreciated that, once the vector has actually been established, used toeffect the coding and then transmitted or recorded for the purposes ofeffecting decoding, the decoding procedure is significantly lesscomputationally demanding. Thus, given a motion vector, it is a verystraightforward exercise to use this vector in order to identify thepreferred block within the transmitted reference frame. However, on thetransmission side, it is necessary to actually calculate the vectorbefore it may be used to effect the coding.

The way in which a motion vector is calculated for a particular blockwill be described with reference to FIG. 2A and FIG. 2B. In FIG. 2A, a16×16 block of pixels to be coded is identified as 25. This block ofpixels is coded by identifying a similar block in a reference frame and,rather then transmitting actual pixel values for the present block,difference values are transmitted which represent the arithmeticdifference between the pixel values in the block of the frame to betransmitted and the pixel values in the identified block of thereference frame.

Block 26 is a 16×16 block of pixels in the reference frame which ispresent at the equivalent position to block 25 in the present frame.Thus, if predictive coding without motion vectors were being employed,pixel values within block 25 would be compared directly with pixelvalues within block 26.

However, when taking account of movement and employing motion vectors, asearch is performed to identify, within the reference frame, a 16×16pixel block which produces smaller difference terms than would beproduced if difference values were calculated from block 26.

In theory, it would be possible to examine all possible blocks withinthe reference frame, however, this would require a massive computationalrequirement and would be virtually impossible to implement in real time.Thus, a compromise is made and, in this particular embodiment, a searchis made by shifting from the base position of the 16×16 pixel block(from the position identified by reference 26) to positions of plus andminus 15 pixel spacings in both the X and Y directions.

Thus, given a plus or minus 15 pixel displacement, a reference block ofpixels for coding block 25 may be obtained from anywhere within block 27of the reference frame. Thus, block 27 is a block made up of 46×46 pixelpositions and a 16×16 pixel block may occupy any of the 961 (31×31)positions within block 27.

In order to determine which of these possible positions provides thebest motion vector, all possible positions are considered and aselection is made, based upon the position which results in the smallestdifference values between block 25 to be transmitted and a referenceblock within block 27. The algorithm executed in order to determine thebest possible motion vector is illustrated in FIG. 2B.

For the purposes of this disclosure, the position of a 16×16 macroblock, such as block 26, within the larger search area, such as block27, will be referred to as the "block positon". Furthermore, the actualpixel positions within a 16×16 pixel macro block will be referred to aspixel position. Block position is described with reference to theposition the block would occupy without being shifted under theinfluence of a motion vector. Thus, the position of a block within area27 is described with reference to the position of block 26. Furthermore,the X, Y co-ordinates of the block position within area 27 are alsoequivalent to the motion vector required to transform block 26 to ablock position at which actual difference values are calculated.

As previously stated, a motion vector is determined for each block byconsidering each possible block position within region 27. This isinitiated by considering the block at the top left corner, positioned bya motion vector (minus 15, minus 15).

At step 30 the next block position is determined, defined by a motionvector (X, Y) which, as previously stated, will be the top left block onthe first iteration.

At step 31 a variable identified as the sum of the differences, sigma D,is set to zero. Thus, step 32 initiates a loop which considers all ofthe pixel positions within the particular block position, thus, at step32 the next pixel position is considered within the block and at step 33an absolute difference value D is calculated by subtracting thereference pixel value from the pixel value to be transmitted, derivedfrom block 25, while ignoring the sign of the difference.

At step 34 the difference value calculated at step 33 is added to thevariable representing the sum of the differences, sigma D, which, on thefirst iteration, will consist of adding the value D calculated at step33 to zero.

At step 35 a question is asked as to whether another pixel position isto be considered which, on the first iteration, will be answered in theaffirmative, resulting in control being returned to step 32.

Thus, at step 32 the next pixel position is identified, the differencevalue for this pixel position is calculated at step 33 and again thedifference value is added to the sum of the differences at step 34.

Thus, all of the difference values for the 16×≠pixel macro block areadded together until all of the pixel positions within the macroblockhave been considered and the question asked at step 35 is answered inthe negative.

In addition to the value sigma D being determined, representing the sumof the differences for a particular block, another variable is stored,identified as sigma D (minimum), representing the lowest modular sum ofdifference values.

On initiating the algorithm, sigma D will have been set to a value largeenough such that any typical value for sigma D will have a smallermodulus, resulting in the question asked at step 36 being answered inthe affirmative.

When the question asked at step 36 is answered in the affirmative, sigmaD (minimum) is set equal to the calculated value for sigma D at step 37.Similarly, at step 38, an X component for the motion vector V (X) is setequal to X, as determined at step 30 and, similarly, the Y component forthe motion vector V (Y) is set equal to Y.

At step 39, a question is asked as to whether another block position isto be considered and when answered in the affirmative, control isreturned to step 30, whereupon the next X, Y block position is selected.

Thus, for each block position, all of the pixel values within that blockare compared against block 25, that is, the block to be coded. All ofthe absolute pixel difference values (that is modulus difference values)are added together and the modulus of this sum is compared to checkwhether it is the smallest found so far. If it is the smallest found sofar, it replaces the previous smallest so far and establishes a new bestmotion vector so far, at step 38.

After all of the block positions have been considered, resulting in thequestion asked at step 39 being answered in the negative, the variablesigma D (minimum) will represents the smallest sum of differences andthe value stored for V (X) and V (Y) will represent the X and Yco-ordinates of the best motion vector. This is the motion vector whichresults in the smallest difference values when the block 25 to betransmitted is compared with a reference block within region 27.

A circuit for performing differential coding is shown in FIG. 3. Aprediction circuit 41 includes a frame buffer for storing pixel valuesof the previous frame. Thus, the previous frame becomes the referenceframe and signals representing differences between the new frame andthis reference frame are transmitted or relayed to the next stage ofcompression, as shown in FIG. 1. Differences are calculated by asubtracting circuit 42, which subtracts outputs produced by theprediction circuit 41 from the incoming pixel values. These pixel valuesare then coded by a coding circuit 43, possibly performing quantisationor Huffman coding, for subsequent transmission or processing.

The output from the coding circuit 43 is also supplied to a decodingcircuit 44, arranged to perform the reverse process to the codingcircuit 43. A decoding process is performed so as to ensure that asimilar reference frame is generated at the originating side as will bereconstructed at the receiving side. It is possible that the codingperformed by circuit 43 will introduce losses, therefore these lossesmust be taken into account at the originating side, to ensure that asimilar picture is re-constructed.

Thus, the output from the decoding circuit 44 is supplied to a summationcircuit 45, the output from which is supplied to the prediction circuit41. In operation, blocks of image data from a frame to be coded areassembled in an input buffer 46. The next block of data to betransmitted is supplied to the prediction circuit 41, which is arrangedto perform the motion vector calculation procedure, detailed previously.As a result of this procedure, the prediction circuit 41 determines amotion vector for the block, which is transmitted or recorded as such.

The prediction circuit 41 stores a complete copy of the reference frame.Initially, it identifies the equivalently positioned block in thereference frame as that for the block presently being coded. From thisequivalently positioned block, the prediction circuit identifies amotion corrected equivalent block, by moving a number of pixel spacings,as defined by the motion vector.

Once a predicted block has been identified by the prediction circuit 41,pixel values for the block being coded are supplied to summation circuit42, which also receives equivalent prediction values from the predictioncircuit 41. Thus, for each pixel in the block being coded, itsequivalent motion corrected pixel is subtracted therefrom and thedifference, produced as an output from circuit 42, is supplied to thecoding circuit 43.

Output pixels from the coding circuit 43 are assembled in an outputbuffer 48, which may be capable of storing several full frames ofpixels, so as to facilitate transmission buffering. That is to say, itis desirable for pixels to be transmitted or recorded at a constantrate, so as to make optimum use of the available bandwidth. However,given the complex nature of the coding procedure, it is difficult togenerate coded pixels at a constant rate, therefore buffer 48 isprovided to smooth out variations in the rate at which coded data isproduced.

As previously stated, the output from coding circuit 43 is supplied toan equivalent decoding circuit 44, whereupon pixel predictions producedby the prediction circuit 41 are added to the decoded values, bysummation circuit 45 and fed back to the prediction circuit 41, tofacilitate the building up of a subsequent reference frame, to allowcoding of the next frame in the sequence.

Thus, it can be appreciated that, given a frame in a particular sequenceof video frames, it is possible to calculate subsequent frames whichoccur in the sequence. This form of coding is referred to as forwardpredictive coding, in that a new frame is coded or decoded withreference to frames which occurred earlier in the sequence. Inparticular, each block of the new frame is decoded with reference to ablock of a preceding frame. Firstly, the block may be in a differentposition from the corresponding position in this previous frame, the newposition being identified by a motion vector. Secondly, transmittedvalues consist of pixel differences between the new block and thepreviously transmitted block.

As described in U.S. Pat. No. 5,083,202, it is possible to code forwardpredictive video signals in real-time, thereby allowing the technique tobe used in transmission systems, in addition to systems where theencoding procedure can take significantly longer than the decodingprocedure.

In accordance with the MPEG video compression recommendation, compressedvideo frames are generated in accordance with the forward predictivemethod described above. Frames of this type are called "P" frames andare one of three types of frame making up a compressed bit-stream.

A requirement of an MPEG system is that it should have entry points,that is to say, reference frames from which a play-back may beinitiated, without making reference to previously transmitted frames.Thus, an MPEG stream may be considered as being made up of groups offrames, wherein each group is substantially self-contained, allowingedit points to be made at the boundaries of said groups. In order toallow image frames to be compressed into groups, it is necessary totransmit a frame which is compressed in such a way that it does notrequire information from any other frames in order for it to beregenerated. Such a frame is said to be intra coded; as distinct frominter coded in which information is required from other frames. Framesof this type are identified in the MPEG standard as "I" frames (intracoded) and P frames are frames that are forwardly predicted from Iframes or other P frames.

A typical sequence of input frames for MPEG coding is shown in FIG. 4. Agroup is identified in this example as consisting of 15 frames although,in accordance with the recommendation, the group number is adjustable,effectively being a trade-off between compression ratio (that is to say,the ratio of compressed data to originating data) and image quality.

In the group of 15 frames shown in FIG. 4A, the intra coded frame I0 isidentified as frame 3. Frame 6 of the input sequence is coded by forwardpredictive coding, as previously described, with reference to the I0frame, frame 3. Similarly, forward predictive coding is also used tocode frame 9, identified as P1, with reference to frame P0. The nextpredicted frame P2 is coded from frame P1 and final forward predictedcoded frame P3 is coded from frame P2. Thus, in a group of fifteenframes there is one intra coded frame, which would be expected torequire significantly more bandwidth than the subsequent frames and fourforwardly predicted frames, derived substantially using the techniqueillustrated with reference to FIG. 3.

The remaining ten frames of the fifteen frame group may be considered asbeing derived with reference to frames which bound them on either side.As far as the actual coding is concerned, frames 4, 5, 7, 8, 10, 11, 13and 14 may be forwardly predicted from an I frame or a P frame, in thesame way in which the P frames are forwardly predicted. Thus, for eachblock, a motion vector is identified, followed by pixel differencevalues for each pixel within the block.

However, it will be appreciated that, on some occasions, a scene changemay occur between a forwardly predicted frame and the reference framefrom which its prediction values are being derived. When this occurs,there is very little correlation between the predicted frame and thereference frame, resulting in very large difference values, which couldtake on similar proportions to the level of information required totransmit the predicted frame as an intra-frame. Thus, under suchconditions, the advantages of the predictive coding are lost, whichwould tend to result in picture degradation as the system attempts totransmit a scene change which requires bandwidth beyond that available.To some extent, such a situation may be tolerated by the output buffer48, assuming other frames are being coded with enough efficiency toprovide additional bandwidth capacity. However, on occasions, it ispossible that changes of this type would result in noticeable picturedegradation.

In an attempt to overcome situations of this type and to further reducethe bandwidth requirement, it is possible that frames 1, 3, 4, 5, 7, 8,10, 11, 13 and 14 are coded with reference to a subsequent frame in thevideo sequence, rather than a previous frame. Thus, rather than thepredictive coding occurring in a forward direction, the predictivecoding is actually occurring in a backward direction. Thus, these framesare identified as B frames (bidirectional) although it should beappreciated that the choice of forward coding or backward coding will bemade on a frame-by-frame basis, depending upon whichever method requiresthe least amount of information. Furthermore, a bidirectional codedframe may be coded with reference to both previous and subsequentframes.

Thus, it may be assumed that a scene change occurs between frame 4 andframe 5. If frame 5 were being generated using forward predictivecoding, its reference frame would be frame 3. Frame 3 forms part of aprevious scene and therefore has little correlation with frame 5, beingtaken from a subsequent scene. Thus, a significant amount of informationis required, if frame 5 is to be faithfully reproduced.

A significant amount of information will have been transmitted, in thisexample, in order to generate frame 6, by the forward predictive codingmethod. It is apparent that a scene change has occurred between frame 3and frame 6 as will be illustrated by the amount of information requiredto generate frame 6, therefore it is clear that the effect of the scenechange has effectively been taken into account in the coding of frame 3and frame 6.

The change occurs between frame 4 and frame 5 and frame 4 can befaithfully reproduced with reference to frame 3. However, when codingframe 5, far less information will be required if said frame is codedwith reference to, say, frame 6, i.e. using the backward predictivemethod, rather than with reference to frame 3. However, the codingsystem does not know that frame 5 should be coded from frame 6, ratherthan frame 3, until attempts have been made to perform both types ofcoding. When both types of coding have been performed, it is thenpossible to compare the results of each and choose the type of codingwhich results in the least amount of data being transmitted.

Thus, in this example both frame 4 and frame 5 will initially be codedin the forward predictive manner and in the backward predictive manner.Then, frame 4 will be transmitted using to the forward predictivecoding, while frame 5 will be transmitted using to the backwardpredictive coding.

Given that a B frame may be generated from an earlier frame, by forwardpredictive coding, or from a subsequent frame, by backward predictivecoding, it is necessary to reorganise the order of frame data fortransmission or recording purposes. This re-ordering is shown in FIG.4B. If it is assumed that frame 1, shown in FIG. 4A, is the very firstframe of a particular sequence, it is not possible to produce this frameby forward prediction, because there are no earlier frames. Thus, bothframes 1 and 2 are produced by backward predictive coding and aretherefore transmitted after frame 3. Thus, as shown in FIG. 5, the firstframe to be transmitted is frame 3, followed by frames 1 and 2. Afterframe 3, it is not possible to transmit frames 4 and 5 because theseframes may also be encoded with reference to frame 6, rather than frame3, therefore the next frame to be transmitted is frame 6, followed byframes 4 and 5. Similarly, the next frame is frame 9, followed by frame7 and 8, whereafter frame 12 is transmitted, the P2 frame, followed byframes 10 and 11, whereafter the P3 frame, frame 15, is transmittedfollowed by frames 13 and 14.

A decoding circuit for an MPEG coded stream is shown in FIG. 5. Thereceived signal is amplified by an amplifier 51, similar to amplifier 20shown in FIG. 1. A demultiplexer 52 separates the pixel information,conveyed in the frequency domain, from the motion vectors and suppliessaid pixel data to a spatial decompression circuit 53. The spatialdecompression circuit 53, similar to circuit 21 shown in FIG. 1,converts the spatially compressed data into pixel values or pixeldifference values, which are in turn supplied to a processor 54.

Processor 54 performs temporal decompression and, in addition toreceiving compressed pixel data from the spatial decompression circuit53, is arranged to receive motion vectors from the demultiplexer 52 andreference data from a video frame store A and a video frame store B. Inresponse to this information, the processor 54 is arranged to generateoutput frames in conventional video format.

As shown in FIG. 4(b), a frame group is transmitted in a re-arrangedorder, such that the decompression circuit 53 receives coded frame 3,followed by coded frame 1, coded frame 2 and coded frame 6 etc. Spatialdecompression is performed and the decompressed frame data is suppliedto the processor 54.

Frame 3 is an intra (1) frame and, as such, will have been transmittedwith only spatial compression. Thus, after spatial decompression hasbeen performed by circuit 53, the pixel data from frame 3 is complete,therefore, no further additional processing is required by processor 54in order to generate output data and the pixel values for frame 3 aresupplied to a selector 55.

However, frame 3 cannot be supplied to the output until frames 1 and 2have been decoded, therefore selector 55 writes the pixel data to framestore A.

As shown in FIG. 4B, the next frame to arrive is frame 1 which, afterbeing spatially decompressed, is supplied to the processor 54. Frame 1was predictively coded with reference to frame 3 (backward coding),therefore output data is generated for frame 1 by reading referencevalues from store A, in response to the motion vectors, and adding thereference pixels to the transmitted pixels. No other frames are codedwith reference to frame 1, therefore selector 55 is arranged to supplydecoded frame 1 to the output.

The next frame to arrive is frame 2. Again, this frame will have beencoded only with reference to frame 3, therefore the processor 54 decodesframe 2 with reference to values read from store A, again in response tothe motion vectors supplied by the demultiplexer 52. Likewise, theselector 55 supplies decoded values to the output.

The next frame to arrive is frame 6, which again has been coded by theforward predictive method, with reference to frame 3. Thus, afterspatial decompression, the processor 54 decodes frame 6, with referenceto the motion vectors and to the reference values stored in store A andsupplies the decoded values for frame 6 to the selector 55. On thisoccasion, frame 6 cannot be supplied to the output prior to frames 4 and5 being decoded, therefore the decoded values for frame 6 are supplied,by the selector 55, to store B. At the same time, frame 3 is read fromstore A and supplied to the output.

Now that decoded versions of both frames 3 and 6 have been stored, it ispossible to decode frames 4 and 5 and to supply these to the output.Frames 4 and 5 may be decoded forwardly, backwardly or bidirectionally,given that all the necessary information is available. After frames 4and 5 have been supplied to the output, frame 6 is read from store B andalso supplied to the output.

After frame 5 has been supplied to the output, it is no longer necessaryto retain frame 3, therefore store A may be overwritten. The next frameto arrive is frame 9. This is decoded with reference to frame 6 andwritten to store A. Thus, the process previously described for frames 4and 5 is repeated for frames 7 and 8. Frames 7 and 8 are decoded andsupplied to the output, whereafter frame 9 can be read from store A anditself supplied to the output.

Thus, it will be appreciated that by following similar techniques, allof the frames can be decoded until another intra frame is received, andthe whole process is repeated. It should also be appreciated that thehardware requirements, particularly in terms of the number of framestores, is not affected by the number of bidirectional frames includedin the frame group. Bidirectional frames are decoded as they arereceived, with reference to the two frames stored in stores A and B.

In applications where the encoding speed is not critical, no particulardifficulties are encountered in encoding a video sequence in accordancewith the MPEG requirements, as illustrated in FIG. 4. However, there aretwo significant problems which arise when attempting to encode inaccordance with this standard, in real-time.

As shown in FIG. 2A, in order to determine a motion vector for just one16×16 pixel macro block 25, it is necessary to consider pixel differencevalues for all block positions within a region 27 (a 46×46 pixelpositions square). Such a region allows suitable motion vectors to bedetermined when predictive coding is being effected between adjacentframes. However, as illustrated in FIG. 4A, it is necessary topredictively code frame 6 with reference to frame 3. In this case, frame6 is not directly adjacent to frame 3 but is separated from frame 3 by atotal of 3 frame spacings.

If movement occurs over the duration defined by frames 3, 4, 5 and 6, itis possible that this movement will take place in a constant direction.Thus, the reference block of pixels from which minimal difference valuesmay be determined, will move across the image frame such that the searcharea will effectively shift on a frame by frame basis. Thus, whereas ashift of plus or minus 15 pixel positions is required for adjacent framepositions, the size of the region must increase as the frame spacingincreases, in order to achieve similar results.

Thus, in order to predictively code frame 4 from frame 3, say, it isnecessary to consider block positions at plus or minus 15 pixellocations, as illustrated in FIG. 2A. However, when predictively codingframe 5 from frame 3, the frame spacing has increased by one frameposition, therefore it would be necessary to consider block positions atplus or minus 30 pixel positions, in order to achieve comparableresults. Similarly, the search region increases by a further plus orminus 15 pixel positions when coding frame 7 from frame 3. Thus, inorder to effect 3 frame spacings, it is necessary to consider a regionof pixels defined by a block displacement of plus or minus 45 pixellocations.

Thus, it will readily be appreciated that the direct calculation ofmotion vectors for frame 6, with reference to frame 3, would result in atremendous computational overhead, if a region defined by displacementsof plus or minus 45 pixel locations were to be searched in accordancewith the algorithm defined in FIG. 2B.

The problem can be appreciated by making reference to FIG. 6. In thisexample, a compression encoding device has received original imageframes 3, 4, 5 and 6, which are to be coded as I0, B1, B2, and P0. FrameP0 is coded by the forward predictive method with reference to frame I0.Thus, all of the frames have been divided into macro blocks of 16×16pixels and each macro block is then coded by defining a motion vector,along with pixel difference values. As previously stated, whenconsidering blocks for coding frame P0, there are three frame spacingsbetween blocks derived from frame I0 and the blocks to be coded withreference to frame P0. In frame 6, block 61 represents a 16×16 macroblock of pixels which are to be coded for transmission as part of the P0frame. Using the forward predictive coding method, the correspondingblock in frame I0 is referenced 62. A motion vector is defined for oneframe spacing, which will be frame 4, which specifies the position ofblock 62 anywhere within square 63. This represents a possibledisplacement of plus or minus 15 pixels in the X and Y directions.

On the next frame spacing, that is from frame 4 to frame 5, a plus orminus 15 pixel displacement must be considered, such that, after twoframes, it is possible for block 62 to be anywhere within square 64.Thus, the area over which difference values are to be calculated hassignificantly increased. However, although a significant increase hasoccurred, it is still not possible to provide a motion vector for frameP0, because this requires yet a further frame spacing.

A further frame spacing results in another plus or minus 15 pixeldisplacements in both the X and Y directions, resulting in block 62being anywhere within square 65. Thus, in order to provide a motionvector for block 61 from block 65, retaining the same level of qualityas if the motion vector had been derived from block 63, it is necessaryto consider pixel displacements up to plus and minus 45 positions inboth the X and Y directions. Clearly, such a level of computation isvirtually impossible to facilitate in real time, using a realistichardware implementation.

As previously described, it is also necessary to bidirectionally encodeframes 4 and 5. Thus, irrespective of whether the forward predictivevalues for frames 4 and 5 are actually transmitted, it is stillnecessary to calculate them, therefore, it will be necessary tocalculate motion vectors for to blocks in frame 4, with reference toblocks in frame 3. This is equivalent to moving block 62 within the box63, as also illustrated in FIG. 2. After all of the pixels in block 62have been displaced by plus or minus 15 pixel positions in the X and Ydirections, it is possible to average these values and calculate apreferred motion vector for the block 62.

An alternative approach known as "telescoping", is illustrated in FIG.7. Block 62 is shown in FIG. 7 as block 72. A motion vector isdetermined by moving block 72 within area 73. As a result of theseoperations, a motion vector is calculated, which effectivelyre-positions block 72 to block 76. Thus, the pixels within block 76 maybe used to forwardly predict a block of pixels in frame 4. Under normalpredictive coding, frame 4 would now be used to predict values for frame5, which in turn would be used to predict values for frame 6.

As previously stated, in accordance with the MPEG recommendation, valuesfor frame 6 are not predicted from frame 5, via frame 4, but arepredicted directly from frame 3. However, given that motion vectors needto be calculated for frame 4 and frame 5 anyway, these motion vectorsmay be used to calculate the final motion vector for frame 6. Thus,rather than trying to calculate a motion vector for frame 6 in one jump,requiring pixel displacements of plus or minus 45 pixel positions, themotion vector for frame 6 is calculated in stages, similar to thearrangement of small segments of a telescope being connected together soas to make quite a long device. Hence, the procedure of deriving amotion vector for frame 6 with reference to frame 3, by calculating amotion vector for frame 4 with reference to frame 3, calculating amotion vector for frame 5 with reference to frame 3, using thepreviously calculated motion vector as an offset and finally calculatinga motion vector for frame 6 with reference to frame 3, again with anoffset, is referred to as "telescoping".

FIG. 7 represents blocks of pixels within a reference frame. A blockwithin a frame to be transmitted is, for the purposes of this example,separated from the reference frame by 3 frame spacings.

Thus, the frames under consideration consist of a reference frame, aframe to be transmitted and two intermediate frames which are positionedbetween the reference frame and the frame to be transmitted.

Each macro block of the frame to be transmitted is consideredindividually. Thus, for a particular macro block, an equivalent blockposition exists in the intermediate frames and in the reference frame.This equivalent block position is identified as block 72 in FIG. 7. Thisequivalently positioned block is compared with an equivalentlypositioned block in the first intermediate frame. A search is nowconducted within a region 73 defined by a plus or minus 15 pixeldisplacement in both the X and Y directions. Thus, the search issubstantially similar to that described with reference to FIG. 2A. As aresult of this search, a block 76 is identified in the reference frameas the block from which pixel values should be obtained in order toreconstitute the block in the first intermediate frame. In addition, amotor vector is indentified which is required to transfer theequivalently positioned block to the position of block 76.

In order to produce a motion vector for the second intermediate frame, afurther search is effected within the plus or minus 15 pixel positionspacings defined by a region 77 centered on the block 76 position.Again, this results in a further motion vector being calculated whicheffectively identifies block 78 in the reference frame. Thus, the motionvector for the second intermediate frame would transform block position72 to block position 78 and the motion vector for the first intermediateframe would transform block position 72 to block position 76.

After calculating the motion vectors for the second intermediate frame,the process is repeated again for region 79 around block 78, againdefined by a plus and minus 15 pixel position spacing. As a result ofthis, a further block position (not shown) within the reference frame isidentified and a third motion vector is produced which transform a blockin the reference frame to a block within the frame to be transmitted,separated by 3 frame spacings. Thus, a single motion vector is producedby telescoping these, such that a block can be transmitted, along with amotion vector identifying the movement of a block from which differencevalues have been calculated, within the reference frame separated by 3frame spacings.

The result of the telescoping procedure is that a single vector isobtained identifying motion of a block in frame 3, which can be used topredict a block in frame 6, without having to do tortuous amounts ofcomputation, as suggested by the illustrated provided by FIG. 6.

A temporal compression circuit for encoding video frames in accordancewith the MPEG recommendation, in real-time, is shown in FIG. 8. Thecircuit shown in FIG. 8 includes an input storage area 81, a referencestorage area 82, a motion vector calculation circuit 83, a spatialcompressor 84, a spatial compressor 85 and a buffering area 86.

In the example shown, the input storage area includes a first inputframe store 87 and a second input frame store 88. The number of framestores actually included in the input storage area depends upon thegrouping of bidirectional frames in the transmitted stream. Thus, inthis example, bidirectional frames are grouped in two's, therefore twostorage devices 87 and 88 are required. However, in accordance with therecommendation, the number of bidirectional frames included in eachbidirectional frame group is variable and it should be understood thatwhen this grouping value increases, a respective increase is required inthe number of input frame stores provided in the input storage area.

The reference storage area 82 includes a first reference frame store 89and a second reference frame store 90. Two frame stores are required inthe reference storage area to allow for bidirectional predictive coding.Thus, the provision of frame stores within the reference storage area isnot dependent upon the type of the actual MPEG stream employed, in termsof the number of bidirectional frames present in each group.

Reference 91 identifies a list of input frames supplied to the circuit,which is equivalent to the order shown in FIG. 4A. As shown in FIG. 4A,the start of a group is initiated by bidirectional frame B-2, followedby bidirectional frame B-1 and then by the group's intra frame I0. Theseframes may therefore be referenced as frame numbers 1, 2 and 3respectively.

When frame B-2 arrives at the circuit, it is not possible to code thisframe because coding is effected with reference to the intra framewhich, at this stage, has not yet arrived. Thus, a selector 92 writesthe input frame data to a first input frame store 87. Similarly, whenframe B-1 arrives (frame 2) it is not possible for this frame to becoded therefore selector 92 writes the pixel information to a secondinput frame store 88.

When the third input frame arrives this will be treated as the I0 frameand, as previously stated, this frame is transmitted without anytemporal compression so that frames for a particular group can beregenerated without reference to any other group. Thus, input selector92 supplies pixel values for the I0 frame directly to an output selector93 which in turn relays said values to a buffering area 86, via an adder94 and a spatial compressor 84.

Adder 94 receives prediction values at its second input from thereference storage area 82. The reference storage area 82 includes aninput selector 95 and an output selector 96, arranged to write data tothe reference frame stores 89 and 90 and to read data from said storesrespectively.

When the I0 frame is being supplied to the spatial compressor 84,selector 96 is effectively disabled, such that no additional values aresupplied to the adder 94 and no predicted values are subtracted from thevalues supplied to said adder from the output selector 93.

At the buffering area 86, pixel values generated by the spatialcompressor 84 are written to a buffer 97 under the control of a writeaddressor 98. Thus, as indicated by reference 99, the I0 frame is thefirst frame written to the buffer 97 under the control of the writeaddressor 98.

In addition to being supplied to the write addressor 98, the pixelvalues for the I0 frame generated by the spatial compressor 84 are alsosupplied to the spatial decompressor 85, effectively performing thereverse function to that of the spatial compressor 84. The output fromthe spatial compressor is supplied to an adder 100 which receives at itssecond input the output generated by selector 96. As previously stated,when processing the I0 frame, no output is generated by this selector96, therefore the output from adder 100, for the I0 frame, is equivalentto the output generated by the spatial compressor 85.

The output from the adder 100 is supplied to the input selector 95 ofthe reference storage area 82 which in turn writes the pixel values tothe first reference frame store 89. The values stored in the input framestores 87 and 88, along with the values stored in the reference framestores 89 and 90, are addressable by a motion vector calculation circuit83. The motion vector calculation circuit 83 is capable of calculatingmotion vectors from four frames which are temporally displaced from areference frame by more than one frame spacing, using the technique oftelescoping previously described. Thus, using these techniques, themotion vector calculation circuit calculates motion vectors for blockswithin the first frame, which is a B-2 frame and for the second framewhich is a B-1 frame. Ideally, as shown in FIG. 4B, the B-2 frame shouldbe coded before the B-1 frame. However, it is not possible to calculatebackwards motion vectors for the B-2 frame until similar backwardsmotion vectors have been calculated for the B-1 frame. This is becausethe motion vectors for the B-2 frame are calculated by a process oftelescoping, therefore the vectors generated for the B-1 frame arerequired as the offset values when calculating motion vectors for theB-2 frame. Thus, the B-1 frame was written to input frame store 88 and,by reading values from this frame store along with values of the I0frame read from reference frame store 89, the motion vector calculationcircuit 83 calculates motion vectors for blocks within frame B-1.

After the motion vectors for frame B-1 have been calculated, it ispossible to encode frame B-1. Given that the circuit is required tooperate in real time, every saving must be taken to ensure that the datais processed as quickly as possible. Thus, being in a position to codeframe B-1 results in the necessary step being taken of actually codingframe B-1. Thus, motion vectors are supplied from the motion vectorcalculation circuit 83 to output selector 96. In response to thesemotion vectors, pixel values are read from reference frame store 89,these being pixel values forming part of frame I0, to produce predictedvalues which are supplied to adder 94. Thus, pixel values for frame B-1are read from input frame store 88 and supplied, via the output selector93, to the adder 94. At the adder 94, the predicted values generated byoutput selector 96 are subtracted from the stored pixel values suppliedby output selector 93 and the differences are then supplied to thespatial compressor 84.

The B type frames do not in themselves provide reference frames for anyother frames, thus, any output generated by spatial decompressor 85 andsupplied to selector 95, does not result in pixel values being suppliedto any of the reference frame stores 89 or 90. However, coded pixelvalues for frame B-1 are supplied to the write addressor 98 which inturn writes these pixel values to the buffer 97.

After frame B-1 has been encoded and the set of related motion vectorshas been stored within the motion vector calculating circuit 83, it ispossible to encode frame B-2, by firstly calculating motion vectors forthis particular frame.

As previously stated, the area of search when calculating motion vectorsis reduced by employing the offset determined for frame B-1 by theprocess of telescoping. Thus, telescoped motion vectors are generated,allowing frame B-2 to be coded with reference to frame I0 which has beenstored in the reference frame store 89. This results in pixel values forframe B-2 being read from input frame store 87 by the output selector 93and being supplied to the spatial compressor 84 via adder 94. At adder94, the predicted values selected by the output selector 96 aresubtracted and the resulting difference values are compressed bycompressor 84. Again, it is not necessary to store any of these pixelvalues as reference values, although said values are supplied to thewrite addressor 98, which in turn writes them to the buffer 97.

After frames B-2 and B-1 have been coded, it is no longer necessary toretain pixel values for these frames, therefore input stores 87 and 88may be overwritten. It will also be appreciated that, at this stage, nodata has been written to reference store 90.

The next frame to arrive in the input sequence, frame 4, is processed asframe B0. Frame B0 may be bidirectionally encoded, therefore it is notpossible to encode this frame until the next B frame has been received.Consequently, frame B0 is written to the first input frame store 87.Input pixel values for frame B0 are also supplied directly to the motionvector calculation circuit 83 which, in real-time, calculates motionvectors with reference to the stored pixel values for the I0 frame. Theforward prediction of frame B0 from frame I0 is calculated and stored,for subsequent comparison with the backward prediction from the next Pframe.

Similarly the next frame to arrive, frame 5, is also a B type frame,resulting in the pixel values for this frame being written to the secondinput frame store 88, under the control of the input selector 92, aswell as being directed to the output selector 93 for the forwardprediction from stored frame I0 as descried with reference to frame B0.

The next frame to arrive, frame 6, is processed as frame P0, a forwardlypredicted frame, therefore input selector 92 directs these pixel valuesdirectly to the output selector 93. Pixel values for frame P0 areencoded by forwardly predicting values from the stored I0 frame. Inputpixel values for the P0 frame are supplied directly to the motion vectorcalculation circuit 83 which, in real-time, calculates motion vectorswith reference to the stored pixel values for the I0 frame, stored inframe store 89, using telescoped motion vectors from frames 4 and 5.

The motion vectors calculated for the P0 frame are supplied to theselector 96, resulting in suitable addressing of reference values fromstore 89 to provide predicted values for the adder 94.

Pixel values for the P0 frame are supplied, from the output selector 93to adder 94 and at adder 94 the predicted values are subtractedtherefrom to provide an input to the spatial compressor 84. The outputfrom the spatial compressor 84 is supplied to the write addressor 98which in turn writes coded pixel values to the buffer 97. At the sametime, the output coded values from the compressor 84 are supplied to thespatial decompressor 85, to effect the reverse process to that performedby the spatial compressor 84.

As previously stated, the output decompressed pixel values from thespatial decompressor 85 are supplied to the adder 100 which receives, atits second input, the predicted values generated by the selector 96.Thus, pixel values are supplied to selector 95 in a form which should besubstantially equivalent to re-constituted pixel values for the P0frame. These pixel values will themselves provide reference values for Bframes therefore said values are written to the second reference framestore 90.

After all of the pixel values for the P0 frame have been considered, itis possible for pixel values for frames B0 and B1, stored in input framestores 87 and 88 respectively, to be encoded using backward prediction.

In producing motion vectors for the P0 frame, with reference to the I0frame, the telescoping process will have been implemented by the motionvector calculation circuit 83. Thus, motion vectors have been calculatedfor frame 4 with reference to frame 3. Similarly, motion vectors havebeen calculated for frame 5 with reference to frames 3 and 4 and againmotion vectors have been calculated for frame 6, the P0 frame, by aprocess of telescoping, with reference to frame 3, 4 and 5. Thus, motionvectors have already been calculated for the B0 and B1 frames in theforward direction. It is now necessary to calculate motion vectors inthe backwards direction, as was done for frames B-2 and B-1.

Again, given that the motion vectors for frame B0 will be calculated bya process of telescoping, it is necessary to calculate the backwardsmotion vectors for frame B1, i.e. frame 5, first.

In order to effect the calculation of these motion vectors, values areread from the second input frame store 88 and supplied to the motionvector calculation circuit 83. A search is then effected with respect tothe reference frame which, on this occasion, will be the P0 frame readfrom the second reference frame store 90. After the motion vectors havebeen calculated, they are supplied to selector 96 which, by readingsuitable reference values from the second reference frame store 90,allows frame B1 to be encoded. Thus, pixel values for frame B1 are readfrom frame store 88 and supplied to the adder 94 via selector 93. At theadder 94, the predicted values generated by output selector 96 aresubtracted from the actual pixel values supplied by selector 93,resulting in difference values being supplied to the spatial compressor84. Again, the spatially compressed values generated by the spatialcompressor 84 are written to the buffer 97 by means of the writeaddressor 98.

Once frame B1 has been encoded, it is now possible to encode frame B0.Again, a search is made within regions of the reference frame stored inthe second reference frame store 90, which on this iteration will be theP0 frame. Telescoping is effected by using the previously calculatedmotion vectors, for frame 5, as offsets to the search. The newlycalculated motion vectors are supplied to the selector 96, allowingpredictive coding to be effected by the adder 94.

Once frames B0 and B1 have been encoded in both the forward and backwarddirection, the motion vector calculation circuit 83 determines, for thebidirectional frames, whether backward predictive coding, forwardpredictive coding or interpolative bidirectional coding should beemployed. This decision is based upon which coding scheme produces thesmallest error. In response to this calculation, selector 96 selectssuitable values from either store 89, store 90 or a combination of thesetwo.

As previously stated, the order in which frames are written to thebuffer 97, in response to the write addressor 98, is identified byreference 99. This order can be compared with the order required underthe MPEG standard, as illustrated in FIG. 4B. In accordance with thestandard, the first frame to be transmitted is the I0 frame, followed bythe B-2 frame, the B-1 frame and the P0 frame etc. However, given thatbackwardly predicted motion vectors for the bidirectional frames havebeen calculated by a process of telescoping, it results that said framesare written to buffer 97 in the wrong order. Thus, as illustrated bylist 99, frame B-1 (frame 2) is written to buffer 97 immediately afterthe I0 frame has been written to said buffer, then followed by the B-2frame. Thus, the B-1 frame and the B-2 frame have been generated in thewrong order.

This ordering situation is resolved within the buffering area 86 bymeans of a read addressor 101. Thus, the read addressor 101 is arrangedto read compressed panel data in a different order from that in which itwas written to the buffer 97. The read addressor 101 is provided withinformation identifying the location of each compressed frame within thebuffer 97. Thus, after information has been written to the buffer 97 inthe order shown by list 99, it is read from the buffer 97 in accordancewith the recommended ordering, as indicated by list 102. Thus, the firstframe to be read from the buffer 97 is the I0 frame which, as previouslystated, was the first frame to be written to said buffer. However,thereafter, rather than the B-1 frame being read from the buffer 97,which was the next frame to be written to said buffer, the B-2 frame isread from the buffer which, as shown in list 99, was actually the thirdframe to be written to said buffer. Thus, the read addressor haseffected the re-arrangement of the frame ordering into that required bythe recommendation.

As shown by list 102, the next frame to be read is the PO frame,whereafter the B0 frame is read in preference to the B1 frame, which isthen read thereafter.

A significant advantage of the system described with reference to FIG. 8is that, although the buffer 86 provides an important role in enablingthe backwards telescopic vector calculation, it does not addsignificantly to the hardware requirements of the system. Buffer 86 isalso necessary because pixel values tend to be written to the buffer ata variable bit rate. The bit rate is dependent greatly upon the inherentredundancy of the image which, obviously, will vary from one videosequence to another. Thus, throughout the video sequence, the actualnumber of bits produced after compression will vary. However, many videoand transmission facilities require a constant bit rate therefore datais read out of buffer 86 at a constant bit rate. Thus, in addition tore-ordering the position of the B frames, the buffer 86 also performsthe important function of absorbing fluctuations in the rate at whichthe data is produced.

A circuit for calculating the motion vectors is shown in FIG. 9. Thecircuit will be described with reference to a telescoping procedure inwhich motion vectors are being determined for a frame separated from areference frame by two frame spacings.

A plurality of processors are provided and when examining an area ofpixels for a two frame spacing, five processors are required. Five suchprocessors are shown in FIG. 9, identified as P1 to P5. Each processordetermines a motion vector for a particular block of the frame to becoded.

The number of times a reference frame is read is minimised bysequentially reading pixels from a region of a reference frame andsupplying the read pixels to all five processors. A region consists of ahorizontal band of pixels but, similarly, it could be a vertical band.Pixels from the region are read and the region is moved by one blockspacing, whereafter pixels in the newly defined region are read againfor the calculation of motion vectors for another group of blocks.

In FIG. 9, motion vectors are being calculated for blocks n-2, n-1, n,n+1 and n+2. If a motion vector were being calculated for block n basedon a one frame spacing, it would be necessary to consider all pixels ofregion 121 of the reference frame. However, given that the blocks beingcoded are separated by two frame spacings from the reference frame, therange of possible pixel positions is that identified by region 122.

As described with reference to FIG. 7, it is not necessary to considerall of the pixel positions within region 122, because a motion vectorhas been calculated for the intermediate frame, which is used to applyan offset to the region of pixels being considered. Thus, the area ofpixels to be considered is equivalent in size to region 121. However,this area can be positioned anywhere within the space of region 122.

The circuit is arranged to process five motion vectors in parallel. Foreach of these, the vertical offset of the window of interest (the sizeof region 121) may be different. Thus, in order to ensure that eachprocessor receives the required pixel values from the reference frame,it is necessary to extend the region read out such that it has avertical dimension equivalent to the height of region 122.

Region scanning for the region illustrated is initiated at pixelposition 123. It extends vertically downwards to the boundary 124 ofregion 122, a distance of five block spacings. The reading of data isperformed in response to addresses generated by an addressor 125, whichis programmed to read regions of the required size, dependent upon thenumber of frame spacings between the frame being coded and the referenceframe.

Reading continues by incrementing the horizontal position by one pixelspacing and again reading pixels for the full five block distance. Thisprocess is repeated until all pixels in the region have been considered.

As pixels in the block n, along with the blocks above and below block nare being read, processor P1 is supplied with pixel values which may berequired to code block n-2. similarly, processor P2 receives pixelvalues which may be relevent to block n-1, processor P3 receives valueswhich may be required for block n, processor P4 receives values whichmay be required to code block n+1 and processor P5 receives pixel valueswhich may be required to code block n+2.

As the scanning moves on, within the region, to the next column ofblocks, including block n+1, all of the pixels required to calculate amotion vector for block n-2 will have been supplied to processor P1.Thus, processor P1 calculates the motion vector for block n-2 and nofurther consideration of this block is required. Processor P1 is nowfree to calculate a motion vector for another block, thus the orderingof the processors, with respect to area positions, effectivelycirculates and processor P1 continues to receive pixel values from thereference frame but now processes these values with respect to blockn+3.

In order to calculate a motion vector, each processor receives fiveblock-columns worth of pixel data. Each processor therefore receivespixel data for an area of pixels five blocks by five blocks square,however, due to the offset defined by the previous motion vector and asa result of telescoping, the processor only needs to process an area ofpixels three blocks by three blocks square. Furthermore, within the timeavailable, it would not be possible to process a five block by fiveblock area.

In order to take advantage of the telescoping procedure, while allowingpixel data from the reference frame to be supplied to a plurality ofprocessors, motion vectors for the previous frame (or frames) aresupplied to an enabling circuit 126. The enabling circuit is arranged tocalculate, for each processor, when it is receiving data from thereference frame which is actually required, in order to calculate thetelescoped motion vector. When such data is being read from thereference frame, an enabling signal is generated and supplied to therespective processor.

As pixel values are been read from the reference frame, some of thesevalues will be required by processor P1. When pixel values are been readwhich are required by P1, an enabling signal is generated by theenabling circuit 126 and supplied to P1 over a processor enable line127. Similarly, if the pixel values being read are required by P2, anenabling signal is supplied to P2 over line 128. Furthermore, similarenabling lines 129, 130 and 131 are provided from the enabling circuit126 to respective processors P3, P4 and P5.

Thus for a processing circuit (P1 etc) to calculate a motion vector fora particular block, it receives pixel values from an area equivalent insize to a five block by five block area. However, the processor onlyprocesses data from an area equal to three blocks by three blocks,although this area could be positioned anywhere within the larger fiveblock by five block area. The smaller three by three block area isdefined by the enabling circuit 126, such that the processors only latchdata, in response to the enabling signal, when pixels are being readwhich are present within the area of interest.

Once a full horizontal region has been read, having a height of fiveblock spacings, a new region is selected by moving the region verticallyby one block spacing. Thus, each row of blocks falls within fiveoverlapping regions (except at the edges of the frame) and each pixelvalue is read five times.

When telescoping over larger distances, more processors are required,given that the area within this the pixels of interest are present maybe larger. Thus, when separated by three frame spacings, sevenprocessors are required. The enabling signal will again identify an areathree blocks by three blocks square, in response to telescoped motionvectors. Thus, as the distance between frames increases, the amount ofdata read out increases but the amount actually processed by theprocessors, while the enabling signal is active, remains the sameproportionally more of the data being ignored due to the enablingsignals being in-active.

I claim:
 1. A method of backward predictive coding frames of a videosignal, said method comprising:generating motion vectors for a frame n-1with reference to a frame n, n and n-1 being integers identifying therelative position of respective frames within a sequence of frames;generating motion vectors for a frame n-2 with reference to a frame nusing said motion vectors for frame n-1, n-2 also being an integeridentifying the relative position of a frame within said sequence offrames; coding frame n-1 by predictive coding using said motion vectorsfor frame n-1; writing coded frame n-1 to a buffer before coding framen-2 by predictive coding using said motion vectors for frame n-2 andwriting coded frame n-2 said buffer; and reading coded frame n-2 fromsaid buffer prior to reading coded frame n-1 from said buffer.
 2. Amethod as in claim 1, wherein a frame n is predicted from an earlierframe n-m by forward predictive coding, m also being an integer and theinteger n-m identifying the relative position of a frame within saidsequence of frames.
 3. A method as in claim 2, wherein frame n-m istransmitted without reference to any preceding or subsequent frames. 4.A method as in claim 2, wherein frame n-m is coded by forward predictivecoding from another frame.
 5. A method as in claim 1 wherein data iswritten to the buffer at a variable bit rate and read from said bufferat a substantially constant bit rate.
 6. Apparatus for predictivelycoding frames of a video signal, said apparatus comprising:means forgenerating motion vectors, said means serving to generate motion vectorsfor a frame n-1 with reference to a frame n and then to generate motionvectors for a frame n-2 with reference to a frame n making use of saidvectors for frame n-1, n, n-1 and n-2 being integers identifying therelative position of respective frames within a sequence of frames;means for coding the video frames by predictive coding using said motionvectors for the frame n-1 and the frame n-2.
 7. Apparatus as in claim 6,including:means for coding frame n by forward predictive coding from aframe n-m, m also being an integer and the integer n-m identifying therelative position of a frame within said sequence of frames. 8.Apparatus as in claim 7, including:means for coding frame n-m withoutreference to any other frame.
 9. Apparatus as in claim 7,including:means for forwardly predictively coding frame n-m from aprevious frame.
 10. Apparatus as in claim 6, wherein said bufferingmeans is arranged to receive coded data at a variable bit-rate. 11.Video conferencing apparatus or video telephony apparatus includingapparatus for predictively coding as in claim
 6. 12. A method ofpredictively coding frames of a video signal, the methodcomprising:firstly generating motion vectors for a frame n-1 withrespect to a frame n, coding frame n-1 by predictive coding using saidmotion vectors and writing it to a buffer, n and n-1 being integersidentifying the relative position of respective frames within a sequenceof frames, secondly generating motion vectors for a frame n-2 bysearching an area offset by the motion vectors for frame n-1, codingframe n-2 by predictive coding using said motion vectors for the framen-2 and writing it to the buffer, n-2 also being an integer identifyingthe relative position of a frame within said sequence of frames, andreading coded frame n-2 from the buffer prior to reading coded frame n-1from the buffer.
 13. A method of predictively coding frames of a videosignal according to claim 12 wherein, prior to generating the motionvectors with respect to frame n, motion vectors for frames n-2, n-1 andn are generated with respect to an earlier frame n-m, where m is aninteger greater than
 2. 14. A method as in claim 13 wherein the framen-m is coded without reference to any preceding or subsequent frame. 15.Apparatus for predictively coding frames of a video signal, saidapparatus comprising:means for firstly generating motion vectors for aframe n-2 by searching an area offset by the motion vectors for framen-1, coding frame n-2 by predictive coding using said motion vectors forthe frame n-2 and writing it to the buffer, n-2 also being an integeridentifying the relative position of a frame within said sequence offrames, and means for reading coded frame n-2 from the buffer prior toreading coded frame n-1 from the buffer.
 16. A method of coding a videosignal in which at least some frames are coded by motion-compensatedinter frame predictive coding, said method comprising:generating motionvectors for a first frame with reference to a reference frame, saidfirst frame preceding said reference frame; generating motion vectorsfor a second frame with reference to said reference frame, said secondframe preceding said reference frame, each such motion vector beinggenerated by comparing a portion of the second frame with portion of thereference frame lying within a search area chosen in dependence on amotion vector generated for the corresponding portion of the firstframe; coding said first frame by predictive coding using said motionvectors for said first frame and writing this first coded frame to abuffer before coding said second frame by predictive coding using saidmotion vectors for said second frame and writing this coded second frameto said buffer; and reading said coded second frame from the bufferprior to reading said first coded frame from said buffer.
 17. Apparatusfor coding a video signal in which at least some frames re coded bymotion-compensated inter frame predictive coding, said apparatuscomprising:means for generating motion vectors for a first frame withreference to a reference frame, said first frame preceding saidreference frame; means for generating motion vectors for a second framewith reference to said reference frame, said second frame preceding saidreference frame, each such motion vector being generated by comparing aportion of the second frame with portions of the reference frame lyingwithin a search area chosen in dependence on a motion vector generatedfor the corresponding portion of the first frame; means for coding saidfirst frame by predictive coding using said motion vectors for saidfirst frame and writing the first coded frame to a buffer before codingsaid second frame by predictive coding using said motion vectors forsaid second frame and writing this coded second frame to said buffer;and means for reading said coded second frame from the buffer prior toreading said first coded frame from said buffer.